Received 17 February 2025, accepted 5 March 2025, date of publication 13 March 2025, date of current version 21 March 2025. Continual Learning with Quasi-Newton Methods STEVEN VANDER EECKT and HUGO VAN HAMME (Senior, IEEE) Department Electrical Engineering ESAT-PSI, KU Leuven, B-3001 Leuven, Belgium Corresponding author: Steven Vander Eeeckt (e-mail: steven.vandereeckt@esat.kuleuven.be).ABSTRACT Catastrophic forgetting remains a major challenge when neural networks learn tasks sequentially. Elastic Weight Consolidation (EWC) attempts to address this problem by introducing a Bayesian-inspired regularization loss to preserve knowledge of previously learned tasks. However, EWC relies on a Laplace approximation where the Hessian is simplified to the diagonal of the Fisher information matrix, assuming uncorrelated model parameters. This overly simplistic assumption often leads to poor Hessian estimates, limiting its effectiveness. To overcome this limitation, we introduce Continual Learning with Sampled Quasi-Newton (CSQN), which leverages Quasi-Newton methods to compute more accurate Hessian approximations. Experimental results across four benchmarks demonstrate that CSQN consistently outperforms EWC and other state-of-the-art baselines, including rehearsal-based methods. CSQN reduces EWC's forgetting by 50% and improves its performance by 8% on average. Notably, CSQN achieves superior results on three out of four benchmarks, including the most challenging scenarios, highlighting its potential as a robust solution for continual learning.INDEX TERMS artificial neural networks, catastrophic forgetting, continual learning, quasi-Newton methods I. INTRODUCTION Since the 2010s, Artificial Neural Networks (ANNs) have been able to match or even surpass human performance on a wide variety of tasks. However, when presented with a set of tasks to be learned sequentially--a setting referred to as Continual Learning (CL)--ANNs suffer from catastrophic forgetting [1]. Unlike humans, ANNs struggle to retain previously learned knowledge when extending their knowledge. Naively adapting an ANN to a new task generally leads to a deterioration in the network's performance on previous tasks. Many CL methods have been proposed to alleviate catastrophic forgetting. One of the most well-known is Elastic Weight Consolidation (EWC) [2], which approaches CL from a Bayesian perspective. After training on a task, EWC uses Laplace approximation [3] to estimate a posterior distribution over the model parameters for that task. When training on the next task, this posterior is used via a regularization loss to prevent the model from catastrophically forgetting the previous task. To estimate the Hessian, which is needed in the Laplace approximation to measure the (un)certainty of the model parameters, EWC uses the Fisher Information Matrix (FIM). Furthermore, to simplify the computation, EWC assumes that the FIM is approximately diagonal.

Tree ensembles are one of the most widely used model classes. However, these models are susceptible to adversarial examples, i.e., slightly perturbed examples that elicit a misprediction. There has been significant research on designing approaches to construct such examples for tree ensembles. But this is a computationally challenging problem that often must be solved a large number of times (e.g., for all examples in a training set). This is compounded by the fact that current approaches attempt to find such examples from scratch. In contrast, we exploit the fact that multiple similar problems are being solved. Specifically, our approach exploits the insight that adversarial examples for tree ensembles tend to perturb a consistent but relatively small set of features. We show that we can quickly identify this set of features and use this knowledge to speedup constructing adversarial examples.

In 2021, cancer was the second leading cause of death in the European Union. Notably, while Europe constitutes only a tenth of the global population, it accounts for almost a quarter of the world's cancer cases, bearing an economic impact of approximately 100 billion annually. Belgian statistics further highlight that every step forward in cancer treatment and care could significantly alleviate the immense personal and societal burden. Globally, extensive efforts are made through a variety of innovative approaches with the ultimate objectives of better prevention, earlier detection, and improved patient outcomes and care. Personalized medicine is considered the holy grail of cancer care in most of these initiatives. Tailoring interventions to the unique characteristics of individual patients promises to revolutionize cancer care. Artificial intelligence (AI) plays a pivotal role in this transformative journey towards precision medicine, aiding researchers and healthcare professionals in accurately predicting cancer risks, enabling earlier diagnoses, and customizing treatment plans to meet individual needs.

Prognostic outcomes related to hospital admissions typically do not suffer from censoring, and can be modeled either categorically or as time-to-event. Competing events are common but often ignored. We compared the performance of random forest (RF) models to predict the risk of central line-associated bloodstream infections (CLABSI) using different outcome operationalizations. We included data from 27478 admissions to the University Hospitals Leuven, covering 30862 catheter episodes (970 CLABSI, 1466 deaths and 28426 discharges) to build static and dynamic RF models for binary (CLABSI vs no CLABSI), multinomial (CLABSI, discharge, death or no event), survival (time to CLABSI) and competing risks (time to CLABSI, discharge or death) outcomes to predict the 7-day CLABSI risk. We evaluated model performance across 100 train/test splits. Performance of binary, multinomial and competing risks models was similar: AUROC was 0.74 for baseline predictions, rose to 0.78 for predictions at day 5 in the catheter episode, and decreased thereafter. Survival models overestimated the risk of CLABSI (E:O ratios between 1.2 and 1.6), and had AUROCs about 0.01 lower than other models. Binary and multinomial models had lowest computation times. Models including multiple outcome events (multinomial and competing risks) display a different internal structure compared to binary and survival models. In the absence of censoring, complex modelling choices do not considerably improve the predictive performance compared to a binary model for CLABSI prediction in our studied settings. Survival models censoring the competing events at their time of occurrence should be avoided.

An artificial intelligence that can predict how a beer will taste from its chemical make-up could help create alcohol-free versions that taste just like regular ones. Predicting flavour from chemical compounds is difficult, as complex interactions between ingredients and the psychology of taste can make for surprisingly different perceptions, even between people sampling the same thing. To address this, Kevin Verstrepen at KU Leuven in Belgium and his colleagues have developed an AI model that can predict flavour profiles based on a beer's chemical components and make suggestions for how to improve the flavour. The model was trained on beer reviews from a panel of 16 expert tasters, who scored each brew for 50 attributes, as well as 180,000 public ratings from an online beer reviewing website. It compared these subjective descriptions with measurements of 226 chemical compounds in 250 Belgian beers.

Whether you prefer a fruity lambic or a complex Trappist, Belgian beers have long been famed for their variety, quality and heritage. Now, researchers say they have harnessed the power of artificial intelligence to make brews even better. Prof Kevin Verstrepen, of KU Leuven university, who led the research, said AI could help tease apart the complex relationships involved in human aroma perception. "Beer – like most food products – contains hundreds of different aroma molecules that get picked up by our tongue and nose, and our brain then integrates these into one picture. However, the compounds interact with each other, so how we perceive one depends also on the concentrations of the others," he said.

Animal X has no legs. Animal X has no legs. Animal X has no legs.

The world's fourth industrial revolution is ushering in big shifts in the workplace. Professor Steven Dhondt has a reassurance of sorts for people in the EU worried about losing their jobs to automation: relax. Dhondt, an expert in work and organisational change at the Catholic University Leuven in Belgium, has studied the impact of technology on jobs for the past four decades. Fresh from leading an EU research project on the issue, he stresses opportunities rather than threats. 'We need to develop new business practices and welfare support but, with the right vision, we shouldn't see technology as a threat,' Dhondt said.

Jeroen Bertels David Robben Robin Lemmens Processing Speech and Images Processing Speech and Images Laboratory of Neurobiology Department of Electrical Engineering Department of Electrical Engineering Department of Neurosciences KU Leuven, Belgium KU Leuven, Belgium KU Leuven, Belgium jeroen.bertels@kuleuven.be Dirk Vandermeulen Processing Speech and Images Department of Electrical Engineering KU Leuven, Belgium dirk.vandermeulen@kuleuven.be In this article, we look into some essential aspects of convolutional neural networks (CNNs) with the focus on medical image segmentation. First, we discuss the CNN architecture, thereby highlighting the spatial origin of the data, voxel-wise classification and the receptive field. Second, we discuss the sampling of input-output pairs, thereby highlighting the interaction between voxel-wise classification, patch size and the receptive field.

One of those masterclasses, covers the topic of "intelligent automation" or as we refer to as "cognitive RPA". Last Tuesday, the first masterclass took place at our own offices in Leuven with over 30 enthusiastic participants. During these masterclasses, we demonstrate the possibilities of combining artificial intelligence (AI) and robotic process automation (RPA). Almost any potential business process for robotic process automation requires some form of human intelligence. Combining the expertise of our venture Brainjar with the people from Roborana, we we're able to deliver an in-depth workshop filled with use cases.